Are we getting interactions wrong?
The role of link functions
in psychological research

Laura Sità, Margherita Calderan, Tommaso Feraco,
Filippo Gambarota, Enrico Toffalini

per chi vuole provare a simulare le cose in tempo reale

qr code che manda a questo link https://github.com/sitalaura/link-functions/tree/main/R

oppure scaricare il file a questo percorso sitalaura.github.io/link-functions/R/datasim.R

1 Example

Simulated dataset 1

independent variable: age in years (years)

dependent variable: (variabile)

Linear model

using the classical linear predictor

fit = lm(y~age, data=d)

Linear model

what we dont see it bc its a default parameter but its actually hidden in our code:

Code 
fit = lm(y~age, family=gaussian(link="identity"), data=d)

the model uses family gaussian and the identity link function

link function in GLMs transforms (re-map) the linear predictor X

to the appropriate range of the response variable Y

2 Example

Simulated dataset 2

independent variable: age in years (years)

dependent variable: mistakes in a TRUE/FALSE task (accuracy)

Linear model

using the classical linear predictor

fit = lm(accuracy~age, data=d)

Linear model

questo modello ci aiuta a predire i dati?

Linear model

no perché i nuovi dati simulati dal modello vanno chiaramente fuori dal range (0,1) di possibili valori per l’accuratezza

❌ Inappropriate model

IN THE FIRST EXAMPLE an identity link was appropriate bc

  • y (boh) spans from -inf to +inf

here an identity link is NOT appropriate bc

  • y (accuracy) spans from 0 to 1

✅ More appropriate model

fit = glm(accuracy ~ age, data=d, family=binomial(link="logit"), weights= rep(k, nrow(d)))

in this case, link="logit" makes sure that y spans from 0 and 1

✅ More appropriate model

3 Studying interactions

Simulated dataset 2

independent variable: age in years (years)

dependent variable: mistakes in a TRUE/FALSE task (accuracy)

adding a new main effect

groups: normal kids (group = 0)

kids with dyslexia (group = 1)

Simulated dataset 2

cosa ho effettivamente simulato

#codice della simulazione

non ho simulato un’interazione, quindi ENTRAMBI i modelli trovano un’interazione che non c’è.

il vero modello in grado di fittare i dati

let’s try out the multiple alternative forced choice (50% - bc of the true/false) probit link

il vero modello in grado di fittare i dati: link=“mafc.probit”

no interaction emerges !!!! as it should

fit = glm(accuracy ~ age*group, data=d, family=binomial(link=mafc.probit(.m=2)), weights= rep(k, nrow(d)))

summary(fit)

Call:
glm(formula = accuracy ~ age * group, family = binomial(link = mafc.probit(.m = 2)), 
    data = d, weights = rep(k, nrow(d)))

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  1.91347    0.03466  55.201   <2e-16 ***
age          0.93311    0.02790  33.441   <2e-16 ***
group1      -0.96005    0.03983 -24.104   <2e-16 ***
age:group1   0.05339    0.03510   1.521    0.128    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 8876.00  on 999  degrees of freedom
Residual deviance:  958.75  on 996  degrees of freedom
AIC: 3040.8

Number of Fisher Scoring iterations: 6

4 Why interactions

Conclusions

Building a model means that we want to find the processo generativo dei dati which, diversamente dal mondo delle simulazioni, we could never know for sure

to do that we must make important decisions

choosing the more appropriate family of distributions to make sure that the new values of the vd im predicting lie within the bounds

choosing the more appropriate link function: otherwise it’s very likely you end up finding non linear effects (ie interactions) that are not there!

We’re conducting a systematic review concerning how often the wrong link functions are used in psychological research + they lead to finding a significant interaction: so far, quite often

Materials & Contact

All materials are available on GitHub at sitalaura/link-functions

Questions and feedbacks laura.sita@studenti.unipd.it

Bibliography

Domingue, B. W., Kanopka, K., Trejo, S., Rhemtulla, M., & Tucker-Drob, E. M. (2024). Ubiquitous bias and false discovery due to model misspecification in analysis of statistical interactions: The role of the outcome’s distribution and metric properties. Psychological methods, 29(6), 1164.

Hardwicke, T. E., Thibault, R. T., Clarke, B., Moodie, N., Crüwell, S., Schiavone, S. R., Handcock, S. A., Nghiem, K. A., Mody, F., Eerola, T., et al. (2024). Prevalence of transparent research practices in psychology: A cross-sectional study of empirical articles published in 2022. Advances in Methods and Practices in Psychological Science, 7 (4), 25152459241283477.

Liddell, T. M., & Kruschke, J. K. (2018). Analyzing ordinal data with metric models: What could possibly go wrong?. Journal of Experimental Social Psychology, 79, 328-348.

Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological bulletin, 105(1), 156.

Thank you

Special thanks to

Supplementary materials

Posterior predictive check link=“logit”

Posterior predictive check link=“probit”

fit = glm(accuracy ~ age, data=d, family=binomial(link="probit"), weights= rep(k, nrow(d)))

Interaction with link=“probit”

a negative interaction emerges

fit = glm(accuracy ~ age*group, data=d, family=binomial(link="probit"), weights= rep(k, nrow(d)))
summary(fit)

Call:
glm(formula = accuracy ~ age * group, family = binomial(link = "probit"), 
    data = d, weights = rep(k, nrow(d)))

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  2.21133    0.03018  73.280  < 2e-16 ***
age          0.81113    0.02295  35.337  < 2e-16 ***
group1      -0.79152    0.03400 -23.279  < 2e-16 ***
age:group1  -0.11299    0.02637  -4.285 1.83e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 8812.26  on 999  degrees of freedom
Residual deviance:  853.32  on 996  degrees of freedom
AIC: 2928.7

Number of Fisher Scoring iterations: 6